arXiv:1504.04432vl [cs.IT] 17 Apr 2015 


On the Cost and Benefit of Cooperation 
(Extended Version) 


Parham Noorzad 
California Institute of Technology 
parham@caltech.edu 


Michelle Effros 
California Institute of Technology 
effros@caltech.edu 


Michael Langberg 

State University of New York at Buffalo 
mikel @ buffalo. edu 


Abstract —In a cooperative coding scheme, network nodes 
work together to achieve higher transmission rates. To obtain 
a better understanding of cooperation, we consider a model in 
which two transmitters send rate-limited descriptions of their 
messages to a “cooperation facilitator”, a node that sends back 
rate-limited descriptions of the pair to each transmitter. This 
model includes the conferencing encoders model and a prior 
model from the current authors as special cases. We show that 
except for a special class of multiple access channels, the gain in 
sum-capacity resulting from cooperation under this model is quite 
large. Adding a cooperation facilitator to any such channel results 
in a network that does not satisfy the edge removal property. An 
important special case is the Gaussian multiple access channel, for 
which we explicitly characterize the sum-rate cooperation gain. 

I. Introduction 

To meet the growing demand for higher transmission rates, 
network nodes should employ coding schemes that use scarce 
resources in a more efficient manner. By working together, 
network nodes can take advantage of under-utilized network 
resources to help data transmisssion in heavily constrained 
regions of the network. Cooperation among nodes emerges as 
a natural strategy towards this aim. 

We propose a network model and use it to study the cost 
and benefit of enabling a cooperation in a given network. 
As an example, consider two nodes, A and B, transmitting 
independent messages over a network M. A third node C that 
has bidirectional links to A and B can help A and B work 
together to achieve a higher sum-rate than they would have 
achieved had they worked separately. 

We seek to understand how the gain in sum-rate resulting 
from cooperation between A and B relates to the capacities 
of the links from (A, B) to C and back. Intuitively, we think 
of the increase in sum-rate as the benefit of cooperation and 
the capacities of the links between (A, B) and C as the cost 
of cooperation. See Figure [T] 

To study this situation formally, let A and B be the 
encoders of a memoryless multiple access channel (MAC). Let 
C be a “cooperation facilitator” (CF), a node which, prior to 
the transmission of the messages over the network, receives a 
rate-limited description of each encoder’s message and sends 
a rate-limited output to each encoder. See Figure 

In one-step cooperation, each encoder sends a function of 
its message to the CF and the CF transmits, to each encoder, 
a value that is a function of both of its inputs. Similarly, 
we can define k-step cooperation (for a fixed positive integer 
k) between the CF and the encoders where the information 



Figure 1. An example of cooperation among network nodes. Node C enables 
nodes A and B to cooperate and potentially achieve higher rates in the 
transmission of their messages over network Af. 


transmission between the CF and each encoder continues for 
k steps, with the constraint that the information that the CF or 
each encoder transmits in each step only depends on the infor¬ 
mation that it previously received. In our achievability result, 
however, we only use one-step cooperation for simplicity. 

Our CF extends the cooperation model introduced by a 
previous work of the authors Q to allow for rate-limited 
inputs. While the CF in 0 has full knowledge of both 
messages and transmits a rate-limited output to both encoders, 
the more general CF we study in this paper only has partial 
knowledge of each encoder’s message. In addition, unlike in 
||T], we allow the CF to send a different output to each encoder. 
We define our cooperation model formally in Section 

The main result of ||TJ states that there exists a discrete 
memoryless MAC where encoder cooperation through a CF 
results in a large gain (with respect to the capacities of the 
output edges of the CF). This implies the existence of a 
network consisting of a MAC with a CF that does not satisfy 
the “edge removal property” 0^0 . We say a network satisfies 
the edge removal property if removing an edge from that 
network does not reduce the achievable rate of any of the 
source messages by more than the capacity of that edge. A 
question that remained unanswered in |[T] was whether such 
a result is true for more natural channels, e.g., the Gaussian 
MAC. The answer turns out to be positive, and except for a 
special class of MACs, adding a CF results in a large sum- 
capacity gain (Theorem [T}. 

Our achievability scheme combines three coding schemes 
via rate splitting. First, each encoder sends part of its message 
to the CF. The CF passes on part of what it receives from each 
encoder to the other encoder without any further operations. In 
this way the CF enables “conferencing” between the encoders, 
which is a cooperation strategy introduced by Willems Q. 

The CF uses the remaining part of what it receives to help 



Figure 2. The network model for the MAC with a CF. 


the encoders “coordinate” their transmissions; that is, it en¬ 
ables the encoders to create dependence among independently 
generated codewords. For this coordination strategy, we rely 
on results from rate-distortion theory ||^ pp. 318-324], which 
were used by Marton |j^ and El Gamal and Van der Meulen 
0 to obtain an inner bound for the capacity region of the 
broadcast channel. 

Finally, for the remaining part of the messages, which do 
not go through the CF, the encoders use the classical coding 
scheme of Ahlswede Q, 0 and Liao p^. We formally 
introduce our achievable scheme in Sectionlffl and study its 
performance in Section [TVl 

In Section |V] we provide an inner bound for the Gaussian 
MAC with transmitter cooperation using methods similar to 
GD- We compare the sum-rate gain of our inner bound with 
the sum-rate gain of schemes that use only one or another of 
our strategies. We see that none alone performs as well as their 
combination, which is the scheme we propose here. 

II. The Cooperation Model 

Let (Afi X 4^2, P(yjxi,X 2 ),y) denote a memoryless MAC. 
Suppose Wi and W 2 are the messages that encoders 1 and 2 
transmit, respectively. For every positive integer fc, dehne [k] — 
{1,..., k}. We assume that Wi and W 2 are independent and 
uniformly distributed over the sets [Mi] and [M 2 ], respectively. 

For i = 1,2, represent encoder i by the mappings 
^ 

U : [M,] X [2”'^“] ^ A-f 

that describe the transmissions to the CF and channel, respec¬ 
tively. We represent the CF by the mappings 

: [2"'^"] X [2"^"] ^ [2"^”‘], 

where denotes the output of the CF to encoder i for 
i = 1,2. Under this dehnition, when {Wi,W 2 ) = {wi,W 2 ), 
the CF receives and (p 2 {w 2 ) from encoders 1 and 

2, respectively. The CF then sends 'tpi{(fi{wi),(p 2 {'W 2 )) to 
encoder 1 and 'tlJ 2 {vi{wi),ip 2 {w 2 )) to encoder 2. 

We represent the decoder by the mapping 
: V” ^ [Ml] X [M 2 ]. 

Then the probability of error is given by 

pW = p{5(r")^(iUi,iU2)}. 

Dehne C“ = and C°“‘ = We 

call the mappings fij 2 , g) an (n, Mi,M 2 ) 


code for the MAC with a (C™, C°'^*)-CF. For nonnegative 
real numbers i?i and R 2 , we say that the rate pair (iii,i? 2 ) 
is achievable if for every e > 0 and sufficiently large n, there 
exists an {n, Mi, M 2 ) code such that Pi”) < e and 

- log Mi > Ri- e, 
n 

for i = 1,2. We dehne the capacity region as the closure of 
the set of all achievable rate pairs {Ri,R 2 ) and denote it by 

Using the capacity region of the MAC with conferenc¬ 
ing encoders 0 (Appendix |^, we obtain inner and outer 
bounds for the capacity region of a MAC with a CF. Let 
'^con{{Ci 2 ,C 2 i) denote the capacity region of a MAC with a 
{Ci 2 ,C 2 i) conference. Since the conferencing capacity region 
can be achieved with a single step of conferencing 0, it 
follows that 

^conf (min{Ci”, Cr*}, min{C‘”, C^^}) 

is an inner bound for '^(C“,C°“*). In addition, since each 
encoder could calculate the CF output if it only knew what 
the CF received from the other encoder, ^conf (C'l”) C^) is an 
outer bound for ‘^(C“,C°"*). We henceforth refer to these 
inner and outer bounds as the conferencing bounds. Note that 
when G™' > G“ and G™' > Cf, the conferencing inner and 
outer bounds agree, giving 

/^out\ _ <v7 //^in /^in\ 

75 ^ ) — ^conf V^l 1^2 )’ 

We next discuss the main result of this paper. For any mem¬ 
oryless MAC (A’l X X2,Piy\xi,X2),y) with a (C“, C™‘)- 
CF, dehne the sum-capacity as 

Gsum — max (^Ri T R2)- 

For a hxed Cin with min{G™, G™} > 0, dehne the “sum- 
capacity gain” G : K>o —>■ R>o as 

G(Gout) = Gsum(C“,C°”*) - Gsum(C“,0), 

where Cout = (Cout) Clout) and 0 = (0,0). Note that when 
Gout = 0, no cooperation is possible, thus 

Gsom(C“,0)= max I{Xi,X 2 -,Y). 

P{xi)P(X2) 

The next theorem states that for any MAC where using 
dependent codewords (instead of independent ones) results in 
an increase in sum-capacity, the effect of cooperation through a 
CF can be quite large. In particular, it shows that the network 
consisting of any such MAC and a CF does not satisfy the 
edge removal property 0^ @- 

Theorem 1 (Sum-capacity). For any discrete memoryless 
MAC (Xi X X 2 ,P{y\xi,X 2 ),y) that satisfies 

max I{Xi,X 2 ;Y)> max I{Xi,X 2 ',Y), 

P{xi,X2) P(xi)P{x2) 

we have G'{0) = 00 . For the Gaussian MAC, a stronger result 
holds: For some positive constant a and sufficiently small 
Gout. 

G(Gout) > ct \/ Gout, 















The proof of Theorem (Appendix |^, is based on our 
achievability result for the MAC with a CF, which we next 
describe. Define 

as the set of all rate pairs that for {i,j) € 

{(1,2), (2,1)} satisfy 

R,<I{Xf,Y\U,Vi,V 2 ,X^) + Cr 
R,<I{Xf,Y\U,Vj,X,) + ao 
Ri + R2 < liXi, X2', Y\U,Vi, V2) + C}” + C™ 
Ri+R 2 < /(Xi, X2; Y\U, V-) + + C.o 

i?i + i ?2 < I{Xi,X 2 ] Y\U) + Cio + C20 

R^+R 2 <I{X^,X 2 -,Y), 

for nonnegative constants Cio and C20, and distributions 
P(u,t;i,U2)P(a:i|u,t;i)P(a;2|M,U2) that satisfy 

C,o<min{Cr,C°"‘} (1) 

/(Vi; F2IC/) < (Cr‘ - C20) + (Cr* - Cio). 

In the above definition, the pair (t/, Vi) represents the 
information encoder i receives from the CF. In addition, the 
pair {Cio,C2o) indicates the amount of rate being used on the 
CF links to enable the conferencing strategy. The remaining 
part of rate on the CF links is used to create dependence 
between Vi and V2- 

Theorem 2 (Achievability). For any memoryless MAC [X\ x 
^2iP{y\xi-,X2),y) with a (C“, C°'^‘)-Cp the rate region 
is achievable. 

A nontrivial special case is the case where the CF has 
complete knowledge of both source messages, that is, C}” = 
= 00. In this case, it is not hard to see (Appendixthat 
C°“*) simplifies to the set of all nonnegative rate pairs 
(i?i,i?2) that satisfy 

i?i </(Xi;y|C/,X2) + Cio 

i ?2 </(^ 2 ; 1 ^|C/,Xi)+C 20 
i?i + i?2 < .^(-^1,-^2; X\U) + Cio + C20 
Ri + R 2 <I{Xi,X 2 ;Y), 

for nonnegative constants Cio < and C20 < Cp*^, and 
distributions P{u, xi,X2) with 

i{Xi-,x 2 \u) < (cr* - C20) + (cr* - Cio). 

Note that in this case, increasing the number of cooperation 
steps (Section]^ does not change the family of functions the 
CF can compute. Thus as with the case where C“ < CJ^* and 
C2" < Cp*^, using more than one step for cooperation does 
not enlarge the capacity region. 

The rate region, C°“‘), in addition to being achiev¬ 

able, is also convex. To prove this, we show a slightly stronger 
result. For every A S ( 0 , 1 ), (C“, C™‘), and (C}", 
define 

= ^(AC“ + (1 - A)C}", AC™' + (1 - A)Cr*). 

Also define C™') and = .;^(C“,C™'). 

We then have the following result. 


Theorem 3 (Convexity). For any A S ( 0 , 1 ), 

2 X^a + (1 - 

The addition in Theorem is the Minkowski sum fT^ , 
defined for any two subsets A and B of as 

A + B = {(tti + 61,02 + 62)1(01,02) G A, (61,62) € Bj. 

If we set C“ = C™ and C™' = C™' in Theoremj^we get 
3 A.^+(l—A).^, which is equivalent to the convexity of 
Using a time-sharing argument, we see that the capacity region 
"^(C'", C°“') also satisfies the property stated in Theorem]^ 
We prove Theorem in Appendix using techniques from 
the work of Cover, El Gamal, and Salehi m 

III. The Achievability Scheme 

In this section, we give a formal description of our coding 
scheme. First, pick nonnegative constants Cio and C20 such 
that Equation (m holds for {i,j} = { 1 , 2 }. In our achievability 
scheme, the first nCio bits of IT} are sent directly from encoder 
i to encoder j through the CE without any modification. We 
thus require C^o to satisfy inequality 'ID- 

Next, choose Cid and C2d such that 
Id S C,;^ - O 20 

^ 2 d S ^2 ~ ^10- 

The values of Cid and C2d specify the amount of 
rate used on each of the output links for the co¬ 
ordination strategy. Einally, choose an input distribution 

P(u, ui, U2)F’(a;i|it,'(;i)P(a;2|'«, ^2) so that P{u, vi,V2) satis¬ 
fies 

C:=Cid + C2d-IiVi;V2\U)>0. ( 3 ) 

Eix e > 0 . Let Ai^^ be the weakly typical set @ p. 521 ] 
with respect to the distribution 

P{u, Ui, U2)P(xi|u, t;i)P(X2|u, f2)P(y|xi, X2). 

By Cramer’s large deviation theorem p. 27 ], there exists 
a nondecreasing function 0 : R+ —)■ M^^uch that 

( 4 ) 

Eix 6 > 0 and let denote the weakly typical set with 
respect to P(u, vi,V2). We make use of the typical sets A^"^ 
and Ae”^ in the encoding and decoding processes, respectively. 

We next describe the codebook generation. Eor i = 1 , 2 , 
let Mi = [2”-^’] and define P^o = min{Pi, C^o}, Rid = 
min{Pi, C™} — Rio, and Ra = (Ri — C'“)+, where for any 
real number x, x'^ = maxjx, 0 }. Note that for i = 1 , 2 , = 

Pio + Rid + Rii, thus we can split each of the messages into 
three parts as 

w, = iWM,W,d,Wu) e [ 2 "^’«] X [ 2 "^-'^] X [ 2 "«“]. 

Here TFio and IU20 are used for conferencing, Wid and lA2d 
are used for coordination, and Wu and W22 are transmitted 
over the channel independently. 


Next, for every (wio,W2o) G generate 

U'^{wio,W2o) i-i.d. with the distribution 

n 

P {c/”Ko, u;2o) = «"} = n 

t=i 

Let E{u^) be the event {[/"(wio, W20) = u^}. Given 

E{u'^), for every {wid, Zi) G [ 2 ”^*“^] x generate 

V^{widTZi\u'^) according to 


V[vr{w,d.z,\u-) = v- 


E{un} = \{P{vu\ut). 

t=i 


for i = 1 , 2 , where P{vi\u) and P(u2|m) are marginals of 
P{vi,V 2 \u). 


Fix (u>io, 1^20, 'M;2d) and functions 

V, : [2"*^"^] ^ V” 

for i = 1 , 2 . Let E{u^, i^i, 1/2) denote the event where 
U'^{wio,W2o) = w” and V{^{wid, .\u'^) = vi{.), and 

V'r{w2d, ■lu"') = V2{.). In addition, for any u'^,iyi, and V2, 
define the set 

A{u'^,Vi,V2) ■= \^{Z1,Z2) ■■ {u^,I^l{Zi),I^2{z2)) G | ■ 
Given E(yE ^vi^V 2 ), if A{u'^ ^vi,V 2 ) is nonempty, define 
vi,V 2 ), ^2(m”, 1^2)) 

as a random pair that is uniformly distributed on A(u^, vi,V2). 
Otherwise, set ZiivP,vi^V2) — 1 for i = 1 , 2 . 


Next, fix (wio, t«20) wid, W2d) and let E{vP, u”, V2) denote 
the event where t/"(wio,W2o) = m", Vi{wid, Zi\u^) = u" 
and V'2"('^2d, ^2|w") = t^2- Por every run and W22, generate 
the codewords X”(wii|rt", ?;") and X^(r(; 22 |w”, r’f) indepen¬ 
dently according to the distributions 



E{u^,v^,v^)] 


n 

Y]_P{Xit\ut,Vit) 

t=l 


for i = 1 , 2 . This completes our codebook construction. 


We next describe the encoding and decoding operations. 
Suppose ILi = {wio,Wid,Wii) and IL2 = {w2o,W2d,W22)- 
Encoders 1 and 2 send the pairs {wio,wid) and {w2o,W2d), 
respectively, to the cooperation facilitator. Thus for i = 1 , 2 , 
(Pi{wi) = {wio, Wid)- The cooperation facilitator then transmits 

'ipi{‘Pi{wi),ip 2 {w 2 )) = {w 20 ,Zi) 
'lp2{‘Pliwi),(p2iw2)) = {wio,Z2), 

to encoders 1 and 2 , respectively. 

Using its knowledge of (rui, W20,-^i), encoder 1 uses the 
(U", U")-codebook to transmit (wii|t 7 ", U"). Similarly, 
using knowledge obtained from the cooperation facilitator, 
encoder 2 transmits 2f^(rt;22|tf"'51^")- 

The decoder uses joint typicality decoding. Upon receiving 
y" the decoder looks for a unique pair {wi,W2) such that 

(u^iwio, W20), V^iwid, Zi), V2^{w2d, Z2), 

X^iwn),X^(w 22 ),y’^) €Ai^l ( 5 ) 


IV. Error Analysis 

In this section, we study the achievability scheme more 
closely and provide sufficient conditions for (i?i, R2) such that 
the probability of error goes to zero. This immediately leads 
to Theorem which characterizes an achievable rate region 
for the MAC with transmitter cooperation. 

Suppose the message pair {wi,W2) is transmitted, where 
Wi = {wio,Wid,Wii). If {wi,W2) is the unique pair that 
satisfies Equation 0 then there is no error. If such a pair does 
not exist or is not unique, an error occurs. We denote this 
event by £. Since directly finding an upper bound on ?(£") 
is not straightforward, we upper bound £ by the union of a 
finite number of events and then apply the union bound. We 
give detailed proofs of the bounds mentioned in this section 
in Appendix 

In what follows, we denote t/”(wio, UI20) and 
Vi^'iwid, ■\U"') by C/" and Ui"(.), respectively. In addition, 
we define 

xr{.)=x-{wu\u\vr{.)). 

We denote instances of 1 G"(.) and 2 f”(.) with and 

Xi(.), respectively. We also write and instead of 

and X^{w^,\U^,V;^). 

We denote the output of the decoder with (r/)i,r(;2). We 
denote U^{wio,W2o) with [/” and similarly define and 
X^ for i = l, 2 . 

We next describe the error events. Eirst, define £q as 

£:o = {(U”,Ui”,U 2 ”)^ 4 ”^}- (6) 

When £0 does not occur, the CE transmits (w2o,Zi) and 
(tuio, Z2) to encoders 1 and 2 , respectively, which correspond 
to a jointly typical triple ({/”, U", 1/2")- Using the Mutual 
Covering Lemma for weakly typical sets (Appendix |^, we 
show that P(fo) goes to zero if C > 4 e, where C is defined by 
Equation Q. 

Next, define £i as 

£■1 = {([/", Vi",U2”,^r, ^2”, U") i 

This is the event where the codewords of the transmitted 
message pair are not jointly typical with the received output 
y". Then we have P(fi\£o) —^-Oasn—>'OoifC< 0 (e)— 4 i 5 . 

If an error occurs and holds, there must exist a message 
pair (wi,W2) different from (wi,W2) that satisfies 0 . The 
message pair (wi,W2), where Wi = (wio, Wid, Wa), may have 
(wio,W2o) 4 ^ lwio,W2o) or (■iuio,W2o) = (wio,W2o)- 

Define £u as the event where {wio,W2o) ^ (wio,W2o)- 
In this case, (E", Vj^, V2^, X^, X2) and y" are independent, 
which implies that P(£[/) goes to zero if i?i -f i?2 < 
I(Xi,X 2 ;V)-C- 7 e. 

If (wio,'W2o) = (wio,W2o), then either (wid,W2d) ^ 
{wid,W2d) or {wid,W2d) = (wid,W2d)- If {wid,W2d) ^ 
(wid, W2d), then wid wid but W2d = W2d, or W2d W2d 
but wid = wid, or wid ^ wid and W2d ^ W2d- 


If such a {wi,W2) doesn’t exist or exists but is not unique, the Let {i,j) G {( 1 , 2 ), ( 2 , 1 )}. If Wid ^ Wid and Wjd = wjd, 

decoder declares an error. we may have wjj ^ wjj or Wjj = Wjj. We denote the former 




event by £viXj and the latter by Sv^- Finally, we denote the 
event where wid ^ wid and W2d ^ W2d with £viV2- 

For {i,j) G {( 1 , 2 ), ( 2 , 1 )}, when SviXj occurs, 
{Vi ,V2 T ^ ^2) and 1"” are independent given 
lu^,V^{.)). This implies V{£v,x,) ^ 0 if 

(i?, - i?,o) + Rjj < I{Xi,X2; Y\U, Vj) - C - 8e. 

For {i,j) G {( 1 , 2 ), ( 2 , 1 )}, when fy. occurs, we 
show that {V(^,V^,Xi,X2] and F” are independent given 
(C/",F,”(.),X”(.)). This implies P{£v^) ^ 0 if R, - R,o < 
IiX,;Y\U,V,,Xj)-C-8e. 

If Sv2V2 occurs, {V{^,V.p,Xi,X2) and F" are indepen¬ 
dent given f/”. Thus P{£viV2) goes to zero if 

(i?i - Rio) + (i?2 - R20) < HXi,X 2; Y\U)-C- 8e. 

Finally, if an error occurs and the message pairs have the 
same {wio,W2o) and the same {wid,W2d), they must have 
different (wii,W22)- We define the events Sxi and 8x1X2 
similarly to the events for {wid,W2d)- The relations 

Xf ^ (C/”, Fi", F2”, Xf) F" 

hold for the events £xi and 8x1X27 respectively. From these 
relations it follows that 

P(£xJ ^ 0 if Ru < /(X,; F| 17 , F, F2, X^) - 4 e 
P(8x,X2) ^ 0 if Rii + R22 < HXi,X2; Y\U, F, F) - 4 e. 

Not surprisingly, these bounds closely resemble the bounds 
that appear in the capacity region of the classical MAC. 

The bounds given in this section can be simplified further 
by replacing Ri — Rio and Ra with {Ri — Cio)^ and [Ri — 
respectively, and noting that the set of all {x,y) that 
satisfy {x — a)+ + {y — b)'^ < c is the same as the set of all 
{x,y) that satisfy x—a < c, y—b < c, and {x—a) + {y—b) < c. 

Note that the general error event £" is a subset of the union 
of the error events defined above. Thus if we apply the union 
bound and choose 6, e, and C, to be arbitrarily small, we obtain 
Theorem | 2 l 

V. The Gaussian MAC 

The Gaussian MAC (T^, |[T^ is defined as the channel 
Yt = Xit + X2t + Zt, where Gaussian 

process independent of (X”,X2) and each Zt is a Gaussian 
random variable with mean zero and variance N. In addition, 
the output power of encoder i is constrained by Pi, that is, 
Sr=i ^ where xu is the output of encoder i at time 
t for i = 1, 2. 

For the Gaussian MAC, we modify the definition of an 
achievable rate pair by adding the encoder power constraints 
to the definition of the {n, Mi, M2) code for a MAC with a 
CF. Then the rate region ^mod is achievable for the Gaussian 
MAC, where ^mod is the same as ^ (Theorem with the 
additional constraints E[Xj^] < for i = 1, 2 on the input 
distribution P(m, ui, V2)P(a:i |u, t;i)P(a:2|M, U2). This follows 
by replacing entropies with differential entropies and including 
the input power constraints in the definition of This is 



Figure 3. The plot of the maximum sum-rate gain by achieved by our scheme 
for the Gaussian MAC with 7^ = 72 = 10^ and = 0.2 as a 

function of Cout- 


possible since we use weakly typical sets |j^ p. 521 ] (rather 
than strongly typical sets) in the proof of Theorem 

If, in the calculation of ^mod, we limit ourselves only 
to Gaussian input distributions, we get a rate region which we 
denote by Note that is an inner bound for the capacity 
region of a Gaussian MAC with a CF. We denote the signal to 
noise ratio of encoder i with ^ and define 7 = 77i72- 
The rate region is given by the next theorem. 

Theorem 4 . For the Gaussian MAC with a (Cin, Cout) CF, 
the achievable rate region is given by the set of all rate 
pairs {Ri,R 2) that for {i,j} = {1,2} satisfy 

^ log(l + pin) + 

^ iog(i + pin) + c^o 

Ri + R2 < - iog(i + phn + ^^72) + c'f + F2” 

Pi + P2 < 2 iog(l + Pili + Pn^j) + + Cjo 

Pi + P2 < ^ log (1 + (1 - Pw)n + (1 - pIo)72 

+ 2poPldP2df) + Fio + F 20 

Pi + P2 < ^ log (1 + 7 l + 72 + 2 (pioP20 + PdPldP2d)i) 

for some pio, P20, Pid, P2d G [ 0 , 1 ], and nonnegative constants 
Fio and C20 that satisfy Equation 0- In the above inequalities 
po. Pa, and pu (for i = 1 , 2 ) are given by 

\ log < (Fr‘ - F20) + (F2°"‘ - Fio) 

Z t Po 

p?i = 1 - Pm - Pid (7) 

~2 _ 1 2 2 2 

Pa ^ PiO PoPid' 

We prove Theorem in Appendix using techniques 
similar to 0). in which the capacity region of the Gaussian 
MAC with conferencing encoders is given. 

Using Theorem]^ we can calculate the maximum sum-rate 
of our scheme for the Gaussian MAC. We define the “sum-rate 












gain” of a cooperation scheme as the difference between the 
maximum sum-rate of that scheme and the maximum sum-rate 
of the classical MAC scheme. In Figure we plot the sum- 
rate gain of our scheme as a function of ( 7 °“* = C™* =: Cout 
for 71 = 72 = 103 , Cl" = Cl" = 0.2 and Cout e [ 0 , 0 . 25 ]. 
We also plot the conferencing bounds in addition to the no 
conferencing sum-rate, which is the sum-rate corresponding 
to a scheme that splits the rate between the coordination 
and the classical MAC strategies and does not make use of 
conferencing (Cio = C20 = 0 ). 

Note that for any value of Cout for which the gain in 
sum-rate is greater than 4 Cout. adding a (Cin, Cout)-CF to 
the Gaussian MAC results in a network that does not satisfy 
the edge removal property. The reason is that if we remove 
the output edges of the (Cjn, Cout)-CF, the decrease in sum- 
capacity is greater than 4 Cout, which implies the decrease in 
either Ri or R2 (or both) is greater than 2 Cout, which is the 
total capacity of the removed edges. On the plot, these are the 
points on our curve which fall above the “edge removal line”, 
that is, the line whose equation is given by gain = 4Cout- 

As we see, the scheme that makes no use of conferencing 
performs well when Cout ^ Qn, and the conferencing scheme 
works well when Cout is close to Cin (and is optimal when 
Cout > Cin). Thus both strategies are necessary for our scheme 
to perform well over the entire range of Cout- In this case 
study, the maximum sum-rate of could have been obtained 
by a carefully designed time sharing between encoders which 
only cooperate through conferencing and encoders that use our 
scheme without conferencing. Whether this is representative of 
our scheme in general (specifically of -^mod and is subject 
to future research. 


VI. Conclusion 


We study the cost and benefit of cooperation under a 
general model introduced in this paper. By adapting the 
coding strategy of Marton ||^ for the broadcast channel to 
the setting of the MAC with cooperating encoders, we suggest 
a cooperation scheme that combines the ideas of ||^ and || 7 ) 
with the conferencing strategy of Willems Q. Based on this 
scheme, we present an inner bound for the MAC with a CF, 
which is sufficient to show a large gain in sum-capacity as a 
result of transmitter cooperation. 


Throughout the proof of our inner bound, we only make use 
of weakly typical sets ||^ p. 521 ] rather than strongly typical 
sets 118 p. 30 ]. This allows the proof of our achievability 
result to go through for the Gaussian MAC without the use of 
quantization. In particular, we present a proof (Appendix of 
the Mutual Covering Lemma 118 p. 208 ] for weakly typical 
sets. 
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Appendix A 

The Coneerencing Encoders Model 

In the conferencing encoders model, introduced by Willems 
0 , each encoder sends partial information regarding its mes¬ 
sage to the other encoder via a noiseless link. The capacity of 
the links going from encoder 1 to encoder 2 and back are 
denoted by C12 and C21, respectively. At every time step, 
each encoder sends information to the other encoder that is 
a function of its own message and what it received from that 
encoder during the previous time steps. After this “conference” 
is over, each encoder transmits a codeword over the channel 
that is a function of its message and information it received 
during the conference. For a blocklength n code, the amount 
of information going from encoder 1 to encoder 2 and going 
back is bounded by nCi2 and nC2i bits, respectively. 

Even though the conference can go on for any finite number 
of steps, the achievability and converse results of Willems 
0 demonstrate that a single step of conferencing suffices to 
achieve capacity. In one-step conferencing, encoder 1 sends a 
rate C12 function of its message to encoder 2 and encoder 2 
sends a rate C21 function of its message to encoder 1 . Then the 
encoders treat the shared messages as a single rate C12 + C21 
common message and use the channel coding strategy of 
Slepian and Wolf GZ)- 

We denote the capacity of a MAC {Xi x 
X2,P{y\xi,X2),y) with a {Ci2,C2i) conference with 
“^conf(< 712 , (721), which is given by the set of all rate pairs 
(i?i,i?2) that satisfy 

Ri<I{Xi;Y\X 2 ,U) + Ci 2 
R 2 <I{X 2 ;Y\Xi,U) + C 2 i 
R i + i?2 ^ I{Xi, X2] Y\U) + C12 C21 

Ri+R 2 <I{Xi,X 2 -,Y) 
for some distribution P{u)P{xi\u)P{x2\u). 

Note that the achievable rate region ^(C‘",C°"*) intro¬ 
duced in Section jl^ satisfies the conferencing bounds. Eor the 
inner bound, if we choose jVij = IV2I = 1 , and 

Cio = min{Ci",C 2 ™*} 

( 72 o = min{C“,Cr}, 

we see that the bounds in the definition of simplify to those 
given by (|^ for C12 = Cio and C21 = C2o- Eor the outer 
bound, notice that the inequalities 

Ri < I{Xr,Y\U,Vi,V 2 ,X 2 ) + C'T 
i ?2 </(X2; rjC/, El, V2, 2 fi) + C” 
i?i + i?2 < I{Xi,X2-,Y\U, El, E2) + ( 7 i" + ( 7 ^" 
i?i+i ?2 </(Xi, 7 f 2 ;E), 

which appear in the definition of are the same as those 
in ^ for C12 = C™ and C21 = C^, since Xi and X2 are 
independent given ((7, El, E2). 

Appendix B 
Prooe oe Theorem[T] 

Eix Cin = { 01 ^, 0 ™) for some positive ( 7 [" and ( 7 ™ and 
let Cout = (< 7001 , Cont)- Define g : ]R>o K>o as 

Il(f 7 out) — < 7 suni(Cin, Cout)- 




Then by Theorem 5(C'out), is bounded from below by the 
maximum of 

min {/(Xi, X2; F|C/, ^1,^2) + C'T + C”, 

/(Xi,X 2 ;y|[/,l/i) + C“ + C 20 , 

/(Xi,X2;y|[/,l/2) + Cio + C“, 

/ (Xi; r I [/, 14, ^2) + / (^2; I [/, , Xi) + Cl 0 + C20, 

/ (Xi, X2; y I c/) + Cio + C20,/(Xi, X2; F)} 

( 9 ) 

calculated over all alphabets Vi, V 2 ), all nonnegative con¬ 
stants (Cio,C 2 o) satisfying Equation (fU, and all probability 
distributions P{u,vi,V 2 )P{xi\u,vi)P{^ 2 \u,V 2 ) that satisfy 

y 2 \U) < 2Cout — Cio — C20- 


The result of the lemma now follows from the nonnegativity 
of mutual information. □ 

If in the above lemma we replace Xi with with 

arbitrary distribution P{vi,V2)P{xi\vi)P{x2\v2) and simplify 
we get 

/(Xi;r|l/ 2 ,X 2 )+/(X 2 ;y|yi,Xi) > I(Xi,X 2 ;Y)-I(Vi;V 2 ), 

since (Vi, V2) —> (-^1,-^2) —>■ F is a Markov chain and 

/(Fi,Xi;F 2 ,X 2 ) 

= H{Vi,Xi) + H{V2,X2) - H{Vi,V2,Xi,X2) 

= /(Fi;F 2 ). 

Therefore, for Cout < 5 (C'out) is bounded from below by 


We next find a simpler lower bound for p(Cout) by evaluat¬ 
ing the minimum in Equation (j^ for fixed alphabets, constants, 
and a special family of distributions. To this end, choose the 
sets U, Vi, and V2 such that \U\ = 1 and Xi C Vt for i = 1 , 2 . 
In addition, let Cio = C20 = 0 and let Pa{xi)Pa{x2) and 
Pb{xi,X2) be distributions such that 

/,(Xi,X 2;F)= max J(Xi,X2;F) 

P{xi)P{x2) 

/ 6 (Xi,X 2 ;F) > 4 (Xi,X 2 ;F). 

Eix vj) G Vi X V2. Eor every A G [ 0 , 1 ], define 
P\ivi,V2,Xi,X2) 

= (1 - A)l{ui = 'yi}l{t ;2 = V2}Paixi)Paix2) 

+ AP&(ui,'U 2 )l{a;i = Ui}l{a ;2 = V2}. 


Eix e > 0 . Consider the equation 


/A-(Vi;F 2 ) + 2 eA* = 2 Cout. 


By Lemma | 7 ] (see end of appendix). 


dCout 

dX* 


= e> 0 . 

A*=0+ 


Thus by the inverse function theorem, there exists a continuous 
increasing function A* = A*(Cout) on [ 0 , 5 i) for some ( 5 i > 0 . 
Thus for Cout < ^1, p(Cout) is bounded from below by 


min {/a.(Xi, X2; F|Fi, F2) + C“ + C“, 

h- (Xi, X2; F|Fi) + C“, /a- (Xi, X2; FIF2) + C“, 

/a.(Xi;F|F 2 ,X 2 )+/a.(X 2 ;F|Fi,Xi), 

/a.(Xi,X 2 ;F)}. 


One of the terms that appears in the above lower bound is 
/a- (Xi ; FIF2 , X2 ) +/a* (X2 ; F I Fi , Xi ). We can further bound 
this expression using the next lemma. 

Lemma 5. For any memoryless MAC, 


/(Xi;F|X2) + /(X2;F|Xi) > J(Xi,X2;F)-/(Xi;X2). 


Proof: We have 

/(Xi; FIX 2 ) + /(X 2 ; F|Xi) + /(Xi; X 2 ) 

= /(Xi;F|X 2 )+/(X 2 ;Xi,F) 

= /(Xi; FIX2) + /(X2; F) + J(Xi; X2IF) 
= /(Xi,X 2 ;F)+/(Xi;X 2 |F). 


min p*(Cout), 
iG{0,l,2.3} 


where 


goiCont) = /a- (Xi, X2; F) - /a. (Fi; F2) 

5i(Cout) =/a*(Xi,X2;F|Fi) + C” 

52(Cout) = /a* (Xi, X 2 ; FIF 2 ) + C” 
gsiCont) = /a* (Xi, X2; F|Fi, F2) + Cj" + C“. 
Note that if Cout = 0 , then A* (Cout) = 0 and 

min pi(Cout) = 5o(0) =/a(Xi,X2;F), 
iG{0,l,2,3} 

since 


Pq{vi,V2,Xi,X2) = l{t;i = VI}1{V2 = V2}Pa{xi)Pa{x2) 

and min{C™,C“} > 0 . Eurthermore, as the gfs are continu¬ 
ous in Cout, there exists a positive 6 smaller than ( 5 i such that 
for every Cout < d, 

min g^{Cont) = 5 o(C'out)- 
iG{0.1,2,3} 

Therefore, for Cout < d, 5(Cout) is bounded from below by 
<?o(C'out) = /a*(Xi,X2; F) - /a.(Vi; F2). 

Since, in addition. 


5 ( 0 )= 5 o( 0 )=/a(Xi,X 2 ;F), 


we have 


g'{ 0 ) > ffo(o) = 


dgo 


dC„ 


C'„ut= 0 + 


dgo 

dX* 

1 


dX* 


A*=0+ dCout 


Cout=0+ 


> -(/b(Xi,X 2 ;F)-J,(Xi,X 2 ;F)), 


where the last inequality follows from Lemma (see end of 
appendix). Since e can be chosen to be arbitrarily small, we 
must have p'( 0 ) = -foo. 


In the special case where our channel is a Gaussian MAC, 
if we choose pio = ^0 = 0 , pid = P2d =■ Pd, and Cio = 
C20 = 0 in Theorem® we see that p(Cout) is bounded from 
below by the maximum of 


rnin Mpo,Pd), 
0<i<4 









calculated over {po,pd), where pd G [ 0 , 1 ], 


Then 


0 < po < \/l - e 4 Cout =: po(C'out)- 


A 

dX 


Ix{Xi;X^) 


= 0 . 

A= 0 + 


The /i’s are defined as 

/o(Po, Pd) = ^ log(l + 71 + 72 + 2poPdV7i72) 
fiipo,Pd) = ^ log (1 + (1 - PoPd)7i) 

+ \ log (1 + (1 - PoPd) 72 ) 

/2(P0, Pd) = 2 log (l + (1 — Pd)7l + (1 ~ PoPd)l 2 ) + (7“ 
faiPo, Pd) = ^ log (1 + (1 - PoPd)7i + (1 - Pd)72) + CT 
f4{po, Pd) = ^ log (1 + (1 - Pd)li + (1 - Pd)72) + <7“ + C)p. 
Next define the function F(po)P3) 

Fipo,Pd) = fo{po,Pd) - rnin /,(po,Pd)- 

l<i <4 

Note that F(1,0) < 0 since 71, 72, C“, and are 
positive. Since F is continuous, there exists Pd >0 such that 
F(1 ,p 2) < 0. However, for any pd, F{.,pd) is an increasing 
function of po- Thus for any po < po(C'out), 

Fipo,p*d)<F{l,p*^)<0. 

In particular, F(^po{Cont), p) < 0. This implies 

p(<7out) > /o(po(C'out),Pd)- 

To calculate fo{poiCout)t Pd) make use of the next lemma. 
Lemma 6 . For constants a and b (b > 0 ) we have 

log (1 + as/ 1 — e~^^) = aVbx + o{^/x) for x > 0. 


Furthermore, if P\{xi,X2,y) = P\{xi,X2)Piy\xi,X2), then 

^ Ja (Xi, X2; r) > /i (Xi, X2; y) - Jo (Xi, X2; r). 

dX A=o+ 

Proof: Note that for every {xi,X2), 
d 


dX 


P\{xi,X 2 ) = Pi{xi,X 2 ) - Po{xi)P{x 2 ). 


Since 




we have 




(Pi(xi.x.) - P„(xOPofe)) log 


^ FA(a;i,a; 2 ) 


Pi(a;i,X 2 ) - Po(a;i)Po(x 2 ) 
T'A(a;i,a; 2 ) 


Pi(a;i) - Po(a;i) Pi{x2) - Poix2) 


P\ixi) 


P\{X2) 


= j;: (p,(xi...) -p,(«)p,(».))log- 


Thus 


dX 


hiXi;X2) 


A= 0 + 


= 0 . 


Proof: We have 

log (1 + as/ 1 — e~^^) = log (1 + as/bx + o{x)) 

— log (1 + as/bx + o{s/x)) 

= as/bx + o{s/x). 

By the previous lemma, 

fo{po{Cont),P*d) - 9 ( 0 ) = ilog(l + - e- 4 C-‘) 

— u s/ f 7 out T o ^ s/ f 7 out) 5 
where p(0) = ^ log(l + 71 + 72) and 

4 pS^V 7 i 72 

a = - --. 

1 + 71+72 

To get the result stated in the theorem, it suffices to choose a 
such that 0 < a < a. The next lemma, used in the appendix 
to calculate the derivatives of /a(Vi;V 2) and I\{Xi, X2',Y), 
follows. 

Lemma 7. Let Poixi)Poix2) and Pi{xi,X2) be joint distri¬ 
butions on Xi X <+2- For every X G [ 0 , 1 ], define the distribution 
P\{xi,X2) as 


For the second part, we write 

h{X,,X2;Y) = H^iY) - H^iY\X,,X2) 

and calculate the derivatives of F[x{Y) and Hx{Y\Xi, X2) 
separately. Note that 

H\iY) = -'^Px{y) log Pa( y), 

y 

thus 

=-'^{^ + '^ogPx{y)){Pi{y) - Po{y)) 
y 

= (^0(2/) - Piiy)) log Pa( y) 

y 

= Po(y) + i 7 i(y) 

+ P(Pi(y)||PA(y)) - P(Po(y)||PA(y)). 
Furthermore, we have 

H{Y\X^,X2) = Y. Px{xi,X2)H{Y\X^ = a:i,X2 = X 2 ), 

X\,X2 

SO 


Px{xi,X2) = XPi{xi,X2) + (1 - A)Po(xi)Po(x2). 


■^Hx{Y\Xi,X2) = Hi{Y\Xi,X 2) - HoiY\Xi,X2). 



Therefore, 


Appendix D 
Proof of Theorem[3] 


^hiXi,X2;Y) = h{Xi,X2;Y) - IoiXi,X2]Y) 

+ D{P,{y)\\P^{y)) - D{Po{y)\\Px{y)). 


Thus 


_d 


h{XuX2;Y) 


A=0+ 


>Ii{Xi,X2;Y) 


Io{X,,X2-,Y). 


□ 


Appendix C 

The Cj" = = oo Case 

In this appendix, we find a simple representation for 
,^(Cin, Cout) (Section]^ in the case where ( 7 “ = C™ = oo. 
If we denote this region with then ^ consists of all rate 
pairs (i?i,i?2) that satisfy 


Ri<I{Xi-,Y\U,V2,X2) + Cio 
R2<I{X2;Y\U,Vi,Xi) + C2o 
Ri + R2<I{Xi,X2;Y\U) + Cio + C2o 
Ri + R2<I{Xi,X2;Y), 

for some Cio < C™* and C20 < Cp*, and some distribution 
P{u,vi,V2)P{xi\u,vi)P{x2\u,V2) that satisfies 

I{Vi; V2\U) < (Cr‘ - C20) + (C2°"‘ - Cio) ( 10 ) 


Fix A G ( 0 , 1 )- Suppose {Ria,R2a) G and 

{Rib,R2b) G ^b- Then there exist constants {Ciq,C2q) and 
{Ciq,C2q) and input distributions 

Pa{u)Paivi,V2\u)Paixi\u, Vi)Pa{x2\u, V2) 
Pb{u)Pbivi,V2\u)Pb{Xi\u,Vi)Pbix2\u,V2) 

that satisfy the constraints of and I^ib for the rate pairs 
{Ria, R2a) and i?2b), respectively. We show that the rate 
pair R2\) G where for i = 1 , 2 , 

Ri\ = Ai?ia + (1 — X)Rib- 


First define 

C^o = AQ + (1 - A)C^ 

for i = 1 , 2 . Notice that by this definition, these constants 
satisfy the constraints of 

Next, consider the input distribution 

P{u')P{vi,V2\u')P{xi\u\ Vi)P{x2\u', V2) ( 12 ) 

where u’ = (rt, s), s G {a, b}, P{s = a) = A, 


and 


P(u|s) 


(Paiu) if s = a 
\Pb(u) if s = 6, 


P{V1,V2\U,S) 


(Paivi,V 2 \u) if s = a 
\Pb{vi,V 2 \u) if s = b. 


Note that this region is contained in the region consisting of 
all rate pairs (i?i,i?2) that satisfy 

Ri<I{Xi;Y\U,X2) + Cw 
R2<I{X2 ;Y\U,Xi) + C2o 
R i + R2<I{Xi,X2;Y\U) + Cio + C2o 
Ri + R2<I{Xi,X2;Y), 

for some Cio < C™* and C20 < Cp*, and some distribution 
P{u, xi,X2) that satisfies 

I{Xi;X2\U) < {cr^ - C20) + - Cio)- ( 11 ) 

This follows from the fact that 


Define P{xi\u',vi) and P{x2\u',V2) similarly. We show that 
(i?ix,f?2A) satisfies the bounds of for this input distri¬ 
bution. Note that for any mutual information of the form 
I{A;B\U,C) we have 

/(A; B\U', C) = \Ia{A- B\U, C) + (1 - X)Ib{A- B\U, C), 


where A, B, and C are arbitrary random variables. Except 
for I{Xi,X2;Y), all the other mutual information terms 
appearing in the definition of are of this form. If we 


compute I{Xi, X2;Y) with respect to ( 12 i we get 


Xi,X2;Y)>I{Xi,X2;Y\S) 


where the inequality holds since 


iU,Vi,V2)^iXi,X2)^Y 


S —>■ (Ail,X2) Y 


is a Markov chain, and any distribution 
P{u,vi,V2)P{xi\u,vi)P{x2 u,V2) that satisfies Equation 
([TU)i also satisfies Equation ([TT]i, since 


IiXi-X2\U)<IiVi,Xi;V2 


irrt 






is a Markov chain. Thus {Ri\,R2\) G and the proof is 
complete. 


Appendix E 

Details of Error Analysis 


To show that these two regions are in fact equal, it now suffices 
to choose Vi = Xi for i = 1, 2, and 


P{U, Vi,V 2 )P{xi\U, Vi)P{x 2 \u, V 2 ) 

= P{u,Vi,V2)5[xi - Vi)5{x2 - V 2 ) 


Before going into the proofs of the error bounds of Section 
IV we need to study the the distribution of our code in more 


detail. Note that 


F’code(w”, Vl,V 2 ,Vi., V 2 , Xi, X 2 , y") 

= P(u^)P(i2ilu^)P(i22lu^)P(v^,v^lu^,,2i,,y2) 

X P(x^lu-, v^)P(x^lu",v^)P(y’^lx’^, x^). 


in the definition of the first region. 






The next lemma relates Pcode{vi ,V2\u^) to the marginals of 
P{vi, t)2 |m"), which is the distribution we use in the definition 
of and 

Lemma 8. For all (u",u”,U2), 

PcodeK,<|u") < 2"(^i-^+^=‘*)PK|u")P(u2"|u”). 


where ( := Cid + C2d - I{Vi] V2\U). 

Next we prove upper bounds on the probabilities of the 
error events defined in Section |IV] 

Bound on P(£o): From the definition of Sq (Equation (j^) 
it follows 


Proof: Note that 

PcodeK,t^ 2 ”k”) ( 13 ) 

= ^ P{v^\u-)P{v2\unPK,vlf\u-,V^,V2). 

We have 

P«,z; 2 "|w",i^i,^ 2 ) < + 0 } 1 {^^ 2 -'K) ^ 0 }, 

where ), for i = 1, 2, is defined as 

y-\v-) = {z,vAz)=v--^}. 

We thus calculate, for i = 1 , 2 , 

Vi 

= i-^p(^,K)i{^-i(ur) = 0 } 

Vi 

= i-(i-p«K))^""" 

< 2 "^-^P«|u"), 


P(£o) = 1 - E ^(«”)^code«,t^2”k”)- 

Let denote the set of all pairs (v^jV^) such that 

(u",u",U2) is in Then 

Y, PiunPcodeK,V^\un 

A'-*) 

= E E ml«”)^(^ 2 |^i”)PK,<|tt^:/l,Z 22 

4 (^) 1^1,1^2 

■^s 

= E^(“”)E 

A'f\u) 

Note that the innermost sum equals 


where the last inequality follows from the fact that 1 — ate < 
(1 — a;)“ for all nonnegative a and x. Therefore, 

PcodeK,t; 2 "l«") < E^(^ll“”)l{^r'W) ^ 0 } 

xYPi^2 \u-)l{yf\v-)^(l}} 

1^2 

and the proof is complete. □ 

Using the next lemma, which relates the value of a 
joint distribution to the values of its marginals, we bound 
Pcode(t'",'^2 1 ^^”) 111 terms of P(u",U2 |m"). 

Lemma 9. For every 

2n{I(Vi-V.2\U)-iS) ^ P(u",U2 Im") ^ 2"(t'(Ti;V2 1 ; 7 )+ 4 ( 5 ) 

“ P(u"|u’^)P(u 2 lu”) ~ 

Proof: For every (u"',u",W2) £ have 

P«,uJ|u") _ P(m")P(u",u",u^) 

P(t;”|M"')P(u 2 |u") P{u^, u”)P(u",U 2 ) 

2-n{H{U)-S)2-n(H{U,VuV2)-S) 

- 2-^(IiiU,Vi)+S)2-n(H{U,V2)+S) 

^ 2n{I{Vi-,V2\U)+4:S) 

The lower bound is proved similarly. □ 

Corollary 10. For all 

PcodeK,t' 2 l«") < 2 "(^+ 4 ^)p«,U 2 ”|u"), 


E PK,v^\u-,yi,y 2 ) 

A'ffur^) 


1 if . 4 .(u”, t2i, 1/2) 7^ 0 

0 otherwise. 


Thus 


E^(«”)^codeK,t; 2 ”l«”) 

a(") 

= E piunpi^iiunpi^'^iunnA^n. 

U'^,Vi,V2 


which implies 

P(£o)= E nw")mk")P(i22|u")lM = 0 } 

u'^ ,Vi ,V2 

= p{^(P",Ui"(.),U 2 ”(-)) = 0 }- 

The last term goes to zero if > 45 (by the Mutual Covering 
Lemma discussed in Appendix |^. 

Bound on P(fi \fo)- Define the set pA) as the set of all 
{vA,Vi,V2,Xi,X2Ty'^) where but 


Then we have 


P(£:i\£:o) = E 

B(") 


U?|zi”) 


1 ; <^2 


X P(a;^|u",u” 


r)P(a:”|u”,t;" 


)P{y-K.xli) 






In this case 


Since the sum is only over all typical triples (rt",u”,), 
Corollary implies 

\£o) < 

X ^ (p(u")PK,U2"|w")P(x^|w",<) 

e(") 

xP(x^|w",U 2 ")P(y"|x^a;^)) 

< 2 ”(^+ 4 ' 5 ) p |(y 4 ("))'=} 

^ 2 "C+ 4 < 5 ) 2 “"®(*^) 


= 5 ] (p(:.i|«")P(:>i|zt") 

1 ^ 1 , 

X P{Vl, V 2 W, Vl,V2)P{v'l, 1)2 1^4", J>1, J^2)) 

= P{v^,v^\u-,V2)P{v^,v^\u-,v^). 

Thus we have the Markov chain 

(ViM/ 2 ",^i",^ 2 ”) ^ (t/",^ 2 ”(-)) ^ 


Thus P(fi \ £0) goes to zero if C < 0 (e) ~ 4 ( 5 . Since 


Therefore we can bound 'P{£y^) from above by 


P(£:oU£i) = P(£:o) + P(£:i\£:o), 

P (f 0 U £i) goes to zero if we choose 5 > 0 and C > 0 such 
that 5 < | 0 (e) and 4(5 < C < 0 (e) — 4 ( 5 . 

Bound on P(£[/): If Ejj occurs, then 
(p", Pi", P2”, ^r, ^2”, Y^) e 

even though (P", l/^",-P",-^2) is independent of 

(P",Pi",P2",Xi",X|‘) (and thus of P") by our code de¬ 
sign. Let Ae"^(P) denote the typical set with respect to 
P{y) and suppose j/" S Ai^\Y). Then let Ai^\y'^) denote 
the set of all (m", u", U2 , a:", ) that are jointly typical 

with y". Then by Theorem 15 . 2.2 of |j^, |Ae"^(y")| < 
2n{H{u,Vi,V2,Xi,X2\Y)+2e)^ Thus P{£u) is bounded from 
above by 


2niR,+R2) ^ P,„de( 2 /”) ^ ^code(^”, <, ^ 1 , ^2 ) ■ 


We now use Corollary 10 to get 


^ Pcode(u”,<,U 2 ",X^X^) 


< 2 "(C+ 4 .) ^ P(w",<,u 2 ",a:^x^) 

Ap’(y") 

< 2"(?+4«) X 2”(^(P''4iW2,^iW2|F)-|-2£) 

^ 2-"(-f4(PVlP2.->fl.^2)-£) 

^ 2-n(I(XuX2-,Y}-C-7e)^ 


Thus P(£:[7) ^ 0 if 


P1+P2 </( 2 fi,P 2 ;P)-C- 7 e- 


Bound on P{£viX2)'- If £viX2 occurs, then 

{wio,W20,W2d) = iwio,W20,W2d), 

but wid ^ wid and 11)22 ^ W22- This implies that there are 
at most 2 ”('^i“'^io)^ and possible values for wi 

and W2, respectively. 


2n((iii-Cio) + -t(fl2-C''") + ) 

xEE Pcode(u", V2, 2/")Pcode(Ui , W 2 ( |u", R 2 ) , 

^(") (22 

We rewrite the sum as 

EE Pcode(u", 1^2, y")Pcode(l’i , , x'l, X^lu^, V 2 ) (14) 

AM (22 

= ^ (p(u")P((r?,x^|7r",<,U2") 

Ap) 

X E -^code (j^2,2/" I w" ) Pcode (u”, U 2 I m" , J22 ) y 

1^2 

Next, we provide an upper bound for the inner sum. 

^Pcode(i^2,y"|u”)Pcode«,U2"|u”,l^2) (15) 

1 X 2 

- E -P(^l|u"')7’code(i^2,y”|u”)Pcode(4'",t;2l'u”,;^l,l^2) 

Pi, 1X2 

< ^ (p(;/i|u")Pcode(i^ 2 ,y"|u”) 

Pl,P 2 

Xl{^ri«)^0}l{^2-'(u2”)y^0}), 

where the last inequality follows from 
Pcode«,U 2 "|ti",^^l,J^ 2 ) < l{^r'«) P 0 }l{z/ 2 -'«) Z^ 0 }- 
From the proof of Lemma we have 

^P(: 2 i|u")l{z 2 ri«) P 0 } < 2 "^i-^P«|u"). 

Pi 

In addition, we have 

^Pcode(i^2,y"k")l{*^2-'(u2”) ^ 0} 

P2 

= P{3z: V,^(z) = U 2 ",P" = y"|P" = u"} 

2"C'2d 

< E P {Y2^(^) = ^2,Y" = y"|P" = w"} 

Z^l 

= 2"C^-P,ode(u2",y”|u”), 

where the inequality follows from the union bound. Thus 

2"(^^‘'+^=^)P«|u")Pcode(u2”(y"|w”) 



is an upper bound for the sum in Equation ( [T 5 ] l. We can now 
bound the sum in Equation ( [T 4 | i from above by 

2'n{Cid-\-C2d) 

< 2"(C+4.) ^ p,„de(w”,t;2",t/")PK,^l,^2l«”,4'2), 

where the inequality follows from Lemma 

Similar to the notation we used to bound P(£’o) 
P{£u), we define Ai’^\u,V2,Y) as the typical set with 
respect to P{u,V2,y)- In addition, for every (M”,U2,y”) € 
Ai'^\u,V2,Y), we define (rt”, uf, y") as the set of all 

(u”,a;”,a;2) such that 


{u^,v^,v^,x^,x^,y^)eAi^ 


Again by Theorem 15 . 2.2 of 0 we have 

A‘f-\u'^,V2,y^) < 2 ”(^(^i’^i’^ 2 l^W 2 .r)+ 2 £)^ 


We can now bound P(fvi) from above by 

2 ™((rti — C'io) 4 +(rt 2 —C' 2 ^) 4 +c+ 4 e) 

X ^ Pcode(u”, U 2 ", y") ^ P«, a;", Iu", U 2 "), 

where the first sum is over all (u",U2,y") in Ai^\u,V2,Y) 
and the second sum is over all K,x 5 *,x^) in 
, y"). We have 


(u'^ ,V2 ,y'^) 

^ 2 n{HiVi,Xi,X2\U,V2,Y)+'2e)2-n{H{Vi,Xi,X2\U,V2)-2e) 
_ 2 -n{HXi,X 2 -,Y\U,V 2 )- 4 :e) 


Thus P{£v^X2 ) ^ 0 if 

(i?i - Cio)+ + {R2 - C“)+ < /(Xi, X2; Y\U, P2) - C - 8 e. 


Bound on P(£vi): When Ey^ occurs, 


We simplify the sum as 

X] X! -Pcode(M”,!^2,X2,y”)Pcode(Xi,uJ,X^,X^|u”,Z22,X2) 

^(") 122 ,X2 

= ^ (PK)PK|U",<) (16) 

X ^ Pcode(j^2,X2,y”|M"-)Pcode(x”,U2,X2|M"',l^2,X2))- 

t^2,X2 

Next, we find an upper bound on the inner sum. We have 
X] ^code ( J^2, X2, y” IW” )Pcode , X 2 , IU”, :/2, X2 ) 

i^2,X2 

= E (^code(^^ 2 ,X 2 ,y”k”)P(i^l|«") 

i^l,i^2,X2 

X Pcode(xr,X2,X2|w”,:2i,Z22,X2)) 

< E (^code(l^2,X2,y”|u”)P(i^l|w") (17) 

i^l,i^2,X2 

X ^ n X 2 '(^ 2 ) 0}), 

where the last inequality follows from 

Pcode (x”, X 2 , X 2 IU”, Z/i, 1/2 , X2 ) 

< iji/r'K) 0 }i{^ 2-'K) n x2-'(2;2) y^ 0 }- 

Erom the proof of Lemma we get 

E^(i^i|u”)i{i^r'«) y^0} < 2”^i-^p(xr|u”). 

In addition, 

E-P™de(l^2,X2,y”|u”)l{j^2'H4'2) nX^Ha^a) 0} 

= P { 3 x : V^{z) = X2”,X2"(z) = x^, r" = y"|P" = u"} 

< E P {P2”W = <,^2”W = ^2,P” = = «”} 

Z^l 

= 2 "^-P,ode(x 2 ”,x^,y"|x"), 

where the inequality follows from the union bound. Thus 


(p"(u;io, u; 2 o), ^i), ^2”^^, ^2), 

X”(rxn|PM>i"),X 2 "(zx 22 |C/M/ 2 ”),P”) £ 4 "^ 

for some wid ^ wid- In this case, (Vi’^,V^,Xi,X2) and 
(P/*, F2”, ^2 ) are independent given (P", V^{.), X^{.)). 

Therefore, 

ivr,V2",x^,x^) ^ (pm 4 "(.),X 2 "(.)) ^ y" 

is a Markov chain. Thus we can bound P{£vi ) from above by 
2 n(it,-C,o )4 ^ E (Pcode(w",^/ 2 ,X 2 ,y”) 

^(") 1/2,X 2 

X Pcode(x”,X^,x 5 ‘,X^|M",J/ 2 ,X 2 )) 


2 ”(^^‘^+^=‘^)p(xr |x”)Pcode(x 2 ”, x 5 , y"|x" 


is an upper bound for the sum in Equation ( [T 7 ) l. We can now 
bound the sum in Equation ( [T 6 | ) from above by 

2 'n,(Cid + C 2 ci) 

X E PcodeK,X2",X^,y")P(xI^|tt”)P(x?|x",xr) 

aA'> 

< E Pcode(x",X2",X^,y")P(xr,X?|u",X2"), 

a'") 

where the inequality follows from Lemma 
Einally, we bound P {£y^) from above by 
2 n((iyi-Cio) ++C+ 4 €) 


X E PcodeK, X”, X^, y") E PKI«", t^2”)PKk”, V 


-'I 


where the first sum is over all (u”, U2, , J/") in 
Ai'^\u,V2, X2,Y) and the second sum is over all (u”,a;") 
in Ae"^(u",U2 ,a;2,t/”). We have 


E 


PK,X?|u”,U 2 ” 


^ 2n{H{Vi,Xi\U,V2,X2,Y)+2e)2-n{HiVi,Xi\U,V2)-'2e) 

^ 2-n{I{Xi-Y\U,V2,X2)-ie) 

Thus P(£yi) ^ 0 if 

(Pi - Cio)+ </(Xi; r|C/, 1/2, X2) - C - 8 e. 

Bound on P(£yjV'2)- The event £viV2 occurs when 
(wio,W2o) = (ti'io,tt'2o), but Wid and W2d ^ W2d- 

In this case, F" is independent of (P”, -^2) given 

? 7 ”. This leads to the upper bound 

2n((iti —Cio)’*' + (-R2 —C'20)’*') 

X ^ Pcode(u", y") ^ Pcode W, ^2 : ^1, ^2 l«”) 
for P(£y^y2), where the sums are over Ai"'\u,Y) and 


Ai"'\u'^,y'^), respectively. By Corollary 


10 


E Pcode«,t^2”,^^^2k” 


we have 
) 




^ 2”(-H‘(^iW2.^l,^2lPV)+2e) 

^ 2 -"(-C+-fi‘(Vi,V 2 .Jfi,X 2 |; 7 )- 6 e) 

^ 2 -"(-f(^i,^ 2 ;V|; 7 )-C- 8 e) 

Hence P(£vjV2) goes to zero if 
(Pi - Cio)+ + (P2 - C2o)+ < /(Xi,X2; r|P) - C - 8e. 

Bound on P(£xi): If £xi occurs, then 
(wio, W20, Wid, W2d, W22) = (wio, W20, Wld, W2d, W22), 
but r&ii ^ wii and 

(P”, Pi", P2”, ^2”. f"") e 4 "^- 

In this case, X” and F" are independent given 
{U'^,Vi,V2,X2)- Thus we can bound P(£xi) from 
above by 


)E^( 




2 niR,-CrY ^ ^ 2 , 2 /" 

where the first sum is over Ai'^\u,Vi,V2, X2,Y) and the 
second sum is over , y"). Further, we have 

^ P{x^\u^,V^) < 2-^(AXuY\U,VuV2,X2)-4e)^ 

(u^ ,v'^ ,V2 ,X2 ,y'^) 

thus P{8xi) —^ 0 if 

(Pi - Ci")+ < /(Xi; F|P, Pi, P2, F2) - 4 e. 

Bound on P{ 8 x 2 X 2 )- When 8 x 1 X 2 occurs, 

{wio,W20,Wid,W2d) = {wio,W20,Wid,W2d), 


but Wii ^ Wii for i = 1,2, and 

(P”, Pi", P2", li", I2", i^”) e . 

In this case F" is independent of (XijX^) given 
(P",Pi",P2"). Thus P{8x2X2) is bounded by 

2 n((i^l-Ci") + + (i^ 2 -C■”) + ) 

X ^ Pcode(«", y”) E ^2 k”, V^) 

where the first sum is over Ai^\u, Pi, P2, F) and the second 
sum is over (m", u", , y"). We have 

^P(a:^a;^|M",<,t;^) < 2 -"(^(-^i'W;VpFiF 2 )- 4 .)^ 


Ap’c 


u"- ,v, ,v; 




thus P{8x1X2) goes to zero if 

(Pi - ( 7 “)+ + (P2 - Cr)+ < /(Xi, F2; F|P, Pi, P2) - 4 e. 
Appendix F 

The Mutual Covering Lemma 

In this appendix, we state and prove the mutual covering 
lemma, which is a variation of a result by the same name in the 
book by El Gamal and Kim fTS) . Our result differs from the 
result in |T^ in two ways. One, our result is stated and proven 
for weakly typical sets, rather than strongly typical sets, and 
two, we require complete independence rather than pairwise 
independence between codewords. 

Lemma 11 (Mutual Covering Lemma). Let U, Pi, and V2 be 
random variables jointly distributed as P(m,Ui,U2). Suppose 
A and B are finite sets with |y 4 | > 2 "’’^ and \B\ > 2 ""^. Given 
jjn _ every {a,b) G A x B, let Pi"(a) and Vf^{b) be 

random vectors generated in an i.i.d. manner according to the 
distributions 

n 

P {Pi"(a) = <|P" = 11"} = J] P{vu\ut) 

n 

P {P 2 "( 6 ) = U 2 "|P" = w"} = [] P{v2t\ut), 

4=1 

where P{vi\u) and P{v2\u) are the marginals of P{vi^V2\u). 
Then 

lim P(3(a,6) gAxB: (P", P"(o), P 2 "(&)) € = 1 

n—^oo L J 

if ri +r2> /(Pi; P2IP) + 4 A 

Our proof, which is given in detail at the end of this 
appendix, follows the achievability proof of the rate-distortion 
theorem given in |j^ pp. 318 - 324 ]. 

The next corollary follows from Lemma in Appendix 
and is the conditional version of Lemma 10 . 5.2 of 0 . 

Corollary 12 . For every (m",?;",142) ^ 

P(t^ 2 |u") > P(i;^|m", 02 -"(^(^ 1 ’T 2 |u)+ 45 )_ 

We next prove the Mutual Covering Lemma. It suffices to 
show 

lim P|v(a,5) gAxB: {U'^Xi {a)Xff{b)) (f = 0. 

n—>-oo L j 




For every (m”, u", ), define 


(Vi,V 2 ), and independently according to the dis¬ 

tributions U ~ A/’(0,1), (Vi,V 2 ) ~ A/’(0,I]), and {X[,X 2 ) ^ 
A/^( 0 ,l 2 ), where 


if (u",<,U2")e4"\ 

]\{U ,Vi,V 2 ) — \ 

I 0 otherwise. 


Then we have 

P 


E = 


1 Po 


{V(a, b) e Ax B: (C/”, V^a), V^{b)) i 4”^} 


X P«|u”)PK|u") 


I-4IIBI 


yPO 1 

and I 2 is the 2x2 identity matrix. Then 


7(14; P 2 IP) = 1 (^ 1 ; P 2 ) = ^l0g^2- 


2 


<^P(w") 1 _ (k{vJ^,v'^,v'^) 


Next, for j = 1, 2, define 


xPK|u")P(i;2"|u",<) 


|Xl||B| 


( 18 ) 




Xi — PioU -l- pidVi + PiiX[. 


where the inequality follows by Corollary By Lemma 
10.5.3 of ||^, which states that for x,y G [0, l]and positive n, 

{l-xy)^ < l-x + e-y^, 

the right hand side of Equation ( [TSl l can be bounded from 
above by 


Note that by Equation (j^, this definition results in = Pi 

for f = 1, 2. Since Y = Xi + X 2 + Z, 

Y = {PiO\/ 1A + P2oV~i^)U + Pld\/^Vi + P2d\fl^V2 

+ pxi^/PiX[ + P22 \/^X2 + Z. 




+ exp (- 1 x ^1 


<1- ^ K{u-,v'l,v^)Piu-,v^,v^) 

u'^ ,-1;” ,^2 

-f exp ( - 

= P{(P",Vi”,L2")^4”^} 

-f exp ( - 

which goes to zero as n —00 provided ri -|- r 2 > 
I{Vi;V2\U)+A5. 


Appendix G 
Proof of Theorem |4] 

Note that if (5, T) is a jointly Gaussian random pair with 
correlation matrix 


Yst = 


( Cr| PCTsCTtA 


\pa-sUT 
then by Theorem 8.4.1 of Q, 

77(5', T) = ^ log 127reEsT | 


4 


ilog((27re)^(l-p2)cr|cr|) (19) 


and 


H{S\T) = H{S,T)- H{T) 

= ilog((27re)(l-p2)CT|y (20) 

Choose {Cio,C2o) and {po, Pio, P20, Pid, P2d) such that the 
constraints of Theorem are satisfied. Then choose U, 


Next, we use Equations ( [TOl l and ( |20) i to calculate the bounds 
in Theorem to obtain Theoremi^ In the calculations that 
follow let {i,j} = {1)2}. We have 


7 (X,; r|P, Vi, L2, Xj) = ff(YlU, El, L2, X,) - 77 (y|Xi, X2) 
= ff(p,,y]^X' + Z)-H(Z) 

= ^iog(i + p4i)- 


Next we calculate 


7(Xi,X2;F|P,Vi,L2) 

= 77(y|P, Li, L 2 ) - H(YIXi,X 2 ) 

= ff(pn \f^iX[ + P22 4427' +Z)- H{Z) 

= 2 +P 11 TI +Pl2l2)- 


Unlike the above calculations, for which we only required 
Equation in the calculation of the next two mutual 

information terms we require Equation ( [20| ), since the ran¬ 
dom variables which appear in the corresponding conditional 
entropies are dependent. We have 


I{Xp,Y\U,V„Xj) 

= H(Y\U,Vj,X^) - HiY\X,,X2) 

= H{p,dy/^V, + pu^/P,X' + Z\Vj) - H{Z) 

= 2 log (1 + ipld + pIH) (1 - 1 + (X+4)7. 


= 2 log(l + pIy) 











I{X^,X2-,Y\U,V{) 

= H{Y\U,V,)-H{Y\Xi,X2 ) 

= + pil^/l\X[ + P22/^^2 + Z\Vi) 

-H{Z) 

= ^ log (1 + pIiz + (1 - p%hj) 

( pWjdij \ 

= \ log(l + Pll^ + P%lj)- 
For the final two remaining expressions, we have 
/(Xi, X 2 ; Y\U) = H{Y\U) - H{Z) 

= \ log (1 + (1 - P?o)7i + (1 - P2o)72 

+ ‘^PoPldP2dl) 

and 

IiXi,X2;Y)=H{Y)-H{Z) 

= 2 l°g (1 + 7i + 72 + 2(pioP20 + PoPidP 2 d)i), 

where 7 = ^^7172. 
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